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ABSTRACT 

Sampled-data control systems are coming of age. Many methods 
have been formulated by which both continuous and sampled-data 
control systems may be optimized. Here one method of optimum 
control design is presented using the approach of dynamic pro- 
gramming. A program to generate the state transition matrices 
from the system dynamics was developed. A program for minimizing 
rather general cost functions was written and examples are pre- 


sented utilizing the results. 
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CHAPTER I 


INTRODUCTION 


In general, any linear control system may be expressed by the fol- 


lowing state equation: 


Eee Du (1) 


[> 
i 


where x; represents the system variables and u,; are the control vari- 
ables. F and D are nxn and nxm matrices defining the dynamical rela- 
tionship of the states and controls. We will sample the states at dis- 
crete intervals t). and at a sampling rate with period T. A recursive 


relationship from the solution of the above differential equation may 


be established: 








x(kt1)= eF lx (k) + oF TT) puck)dt (2) 
® 

where 2 m2 Fn _n 

Fls 1 } Vee = seas rt eres (3) 
; ie 
Equation (2) may be written: 
x(kt+1)= ox(k) + Auk) (4) 
where > = afl (5) 
Glee 

and A “i e ( Ddt (6) 


As an example of the above equations, the open-loop system of 


Figure I-1 will be considered. 





aD l/s xy 





Figure I-1l: Block diagram of an open loop control system. 


By inspection, the following state equation defines the example sys- 


tem: 


|< 
Il 
|x 
+ 
I< 


(7) 


The matrix as defined by (5) has only two non-zero terms in the 


Taylor expansion for the exponential function: 


1 0 1 
b= al T (8) 
0 i 10 0 
or 
1 iT 
oS (2) 
0 i 
In a like manner, the convolution integral of (6) may be solved 
fOr) Ae: 
Tack T-T Q 
A = at (10) 
O 0 it ak 
giving ; 
E 
A= 1) Ci) 
L 


The system difference equation may now be written as a function 


of the sampling period T: 


2 
i i il 
x(ktl) = x(k) + Va u(k) (12) 
Q 1 Al 


The calculation of the transition matrix, $, and the external 


forcing function's impulse response matrix, A, from the matrices F and De 





of the process dynamics was accomplished with the digital computer. This 
program required an input of F and the sampling rate T. The amount of ac- 
curacy desired may be varied by the user by inputing the least significant 
figure desired. This test point for accuracy results from the fact that 
the sum of all terms in a Taylor series following the test term is less 

in magnitude than the test term. Figure I-2 is the flow chart of the 
program. The program has been incorporated into other programs written 


and presented in this paper. 





START 
Nw 


READ: 
NDE tobe ata 1) 


TELM= TERM*DT 
DELP= TELM 


DELM= 0.0 












TM= TM+1.0 
DELM= DELM - TELM*F*DT/ (TM+1.0) 
WORM=WORM+TERM*F DT /T™ 


TERM=WORM DELP=DELP+TELM 


WORM= 0.0 DELM= 0.0 
TELM=DELM $= o+TERM 


ABC = | raRM CRTC) | max 





TEST: ABC 


¢ > 


A= A+o*DELP*D 
PRINT: @¢ and A 


7X 


STOP 


Figure I-2: Flow Chart for PROGRAM PHIDEL 





The optimal control of a system depends upon a valid determination 
of an optimal control policy which will maximize or minimize a perform- 
ance criteria. This control policy may be generated from criteria in- 
volving time, system state variables, noise, control effort, etc. In 
some cases, this control policy may be generated by active networks or 
filters incorporated within the system, or a digital computer may be 
used to generate the control. To accomplish this, all dynamics of the 


system must be fed back to the digital computer as indicated in Figure I-3. 





Figure I-3: Multivariable Digital Control System 

A general approach to the optimization of any dynamic system was 
developed through the efforts of R. E. Bellman and has been called 
‘dynamic programming’. As an example of this method, in a missile prob- 
lem, one might choose a proper cost function (terminal CEP) and minimize 
this cost throughout the trajectory. The optimum trajectory, according 
to the chosen cost function, would be computed. 

Dynamic programming appears to be an excellent method for determ- 
ining an optimal control process because of its adaptability to use on 
a digital computer. Throughout the remainder of this paper, this tech- 
nique will be utilized in formulating general programs to synthesize 


optimum controllers. 


— = Seti a 





CHAPTER Il 


PROGRAM OPCON1 


Again let us consider the system: 


x =Fx+Du | (1') 


whose discrete Solution is 


x(kt+1)= $x(k) + Au(k) (4°) 


where we now take the case where u(k) is a scalar control and A is 

a column vector of system impulse responses due to control inputs. 

The recursion formulas will be derived for this system minimizing the 
sum of the quadratic terms as described by the matrix Q over an N stage 
process. The Q matrix is specified by the requirements of the user and 
the application of solid engineering judgment. This cost function may 
be written: 


J(N)= minimum 
k 


toms 


x(k) Qx (k) is) 
: . 


By starting with the last stage first, equation (7) becomes: 


N 
J(N)= minimum { x7 CN) Que ()+ ay x () x Ce} 


or 


J(N)= minimum {xT (nyoxcny} + J(N-1) (14) 


It will now be noted that only the x? (N)Qx(N) term contains the control 
policy u(N-1); therefore, to minimize this function over the last stage, 


we need only to find: 





9 (xc (N) Qc (N) )_ 
9 u(N-1) : ey 


By substitution of equation (4) into x (N)Qx(N): 


xT (nyauin)= [ox(v-1)+4u0-1)] “9 [ox -1)tauw-1)] (16) 
and 
9(x- (N)QA(N)) a T 
ce) = 2A Qox(N-1)+ 2A°QA u(N-1)=0 (17) 
OL 
alo 
*, u(N-1)= - ATQA x(N-1) (18) 


Equation (18) now represents the optimum control policy u(N-1). It 
will be noted that we are dividing by a multiplication of three matrices; 
this is a valid operation because a row matrix times a square matrix 
times a column matrix will always yield a scalar number if the multi- 
plication is valid. The multiplication is valid if the dimensions of 
the respective matrices are proper; and in Eeeieee it will always be 
valid. 

By proceeding back one more stage, the control policy u(N-2) may 
be optimized. | 

J(N)= minimum jx! (N) Qx (N) +x? @-DeRaHDf +J (N-2) (19) 


Again by direct substitution, equation (19) becomes: 


T 
J(N)= cinimun} & ty, Poe (8-2)4hu(N-2) "a 
ANA 


{ gAM RS se av-2)t0u (NN 23 
a QA (20) 


i frxor-2yenuara) | ae [ax (t-2)-40u 0-2) | +J (N-2) 





Minimizing with respect to u(N-2), remembering that J(N-2) is not a 
N- 
function of u(N-2); and OJ (N- Yes (N- 2) 


Cray = 2A Qox(N-2)+24 QAu(N-2) 


Tomo sp o¢ 
AA~ Qo pee ae a 
+2AT{¢ ae o> x~ (N-2)Q{¢ ATG JA 


al 7, 
AA AAtod 
26 ag, IM 9 FTO, u(N-2)=0 (21) 
i 
v= |r GH 52 (QtPa) Hig gc P= (22) 
AT (QHP . )A 2 

therefore 

J (N) 


Jyu(N-2) = A°Qox(N-2)+0" V5 Q Hy Ox (N-2) 


+A Qau(N-2)+A* ¥7QAu(N-2)= 0 (23) 
oo AT (QtyTQu,)¢ 
oe N=2 =e 2 Eee N=2 24 
aT (Qtyt Qu, ya Ecc N= 2) (24) 
Wee P= vou (25) 
ii 1 


(26) 


The optimum control policy for the (N-2) stage has been found represent- 


ed by equation (26). 


To further the establishment of recursion formulas, the next stage 





back must be considered and minimized with respect to u(N-3). The fol- 


lowing symbology will be utilized in this stage of the development: 


A= [ox v-3)44u0N-3)| (up 
and 
+P 
ie e bA+y a = “( ——\ (28) 
ite (Q+P, dA 


The cost function then becomes: 


J(N) = minimum (atQa+B QB }4+J (N-3) (29) 
Let vo = | I- AAT (Q+P,) |e (30) 
A" (Q+P,)4 
Therefore a T 
ean ee Z 
J(N)= minimum{A*QA+ ies Q [v1 ¥ 24] }4+J (N-3) (31) 


Minimizing the cost function with respect to u(N-3): 


9I(N) aT . T 
Du (N-3) 2A~ Qox (N-3)+2A~ QAu (N-3) 


T it 
128 ¥) iat) 


+2079", "Qh, Vodu(N-3) = 0 (32) 
Solving for u(N=3): 
at Q¢tAy. v. QU, ¥59 
UO eee (33) 
ATQAFAT YS YT 64 vod 





giving 


ic Took 
u(n-3) = — 2 (Qt¥D¥, QW, ¥,)¢ ‘3 
T (pemne % 
A (QH5 07 QU ¥)4 


Let 


Therefore 


so5)) 


(>) 


(36) 


It is believed that an iterative relationship may now be shown without 


regressing to another stage. The following equations may readily be 


shown to be true by an inductive proof. 


_ i _ Bd" (QHP,) 


Vietl = 
AT(QHP,) 
where 
aT, 0 Te 
Pe YY ed ere 
OT “i 
Bg) 7 
Pr = a Ke? Py 0 
and 
A” (Q4P,_1) ¢ 
u(N-k) == (eee x (N-k) 
Aaa D a 
x(k+1) = $x(k)-+Au(k) 


(37) 


(38) 


)), 


(40) 


(41) 


To illustrate the use of the above equations, the system presented 


in Chapter I will be considered with the sampling period equal to one 


second: 1 1 Le 1 0 
o = a= Q= 
0 l il 0 0 


10 


To find u(k), equations 37, 39, and 40 must be solved: 





T 0 0 
¥, =o - SEM X (42) 
AtQa -2 =] 
and 
Pp. = ploy, = 0 (43) 
1 _ oe 
and 
A 
Se u(k)= - SOP (ey = -25 (k)=2%9 (K) (44) 
4 Qa 


By substitution into equation (44) and solving for u(0), we find 


x, (1)=0 and Koil) = -2. The following phase plane (Figure II-l) results: 





Figure II-l: Phase Plane For Example Problem 


This clearly indicates that a limit cycle reaee ee that the cost func- 
tion, as determined with the use of this weighting matrix, yields a rela- 
tive minimum cost of one unit. 

Using the derived recursion formulas 37, 39, 40 and 41, PROGRAM 
OPCON] was written. Figures II-2 through II-10 represent abbreviated 


flow charts of the program and the subroutines used in the program. It 


ee 





must be remembered that the arithmetic operations represented in the 

flow charts are all matrix operations. As can be seen from Figure II-2, 
the required inputs to the program must be read from data cards. The 
following table (Table II-1) indicates the order that the data cards are 
read, the format required on these cards, and the definition of the vari- 
able concerned. The primary output of OPCON1 is the value of each state 
variable, the control signal, and the cost of each stage. A complete pro- 


gram listing may be found in the appendix. 


me: ae Oe 


Format of 
Variable Definition Data Card 






































M Number of stages plus 5 
one 
N Order of system iS 
Column vector of 8F10.6 
system impulse (read from top of 
J responses due to column to bottom) 
control inputs 
State transition 8F10.6 
g matrix (read in order by 
columns ) 
Weighting matrix of 8F10.6 
Q the cost function (read in order by 
columns ) 
Initial conditions 8F10.6 
x (0) on the state vector (read from top of 


column to bottom) 


OP oe Oi Re 


Table II-1: Definition and Format of Data Cards for Program 
OPCON1. 


1 2 





START 





PRINT: 
DATA CARDS 








Zee) — O(1, J) 
77(i,d)= Z(1,1,J) 


ol 


Figure II-2: Flow Chart Of Program OPCON1 


12 





al 


FEND: 
PSI,P,and (QtP) 


a2 


FIGURE Ii-—2: 


14 


P(I,J,K)= PJ(J,K) 
p(I,J,K)= RR(J,K) 
Z(I+1,J,K)= ZZ(J,K) 


> 
i % 
o < 


(continued) 












a2 


PRINE: 
P,W,and(QtP) 





PRINT: 
INVERTED (QtP) 


© 


a3 


Breure Ti-Z- (continued) 


b 


ee 





J> 
a 


a3 


> 


K>D 


SMM (J, K)= sM(1,J5%) | 


u(1)= H*x(T) 


x(Itl)= o*x(1)+A*u(I) 


FIND: 
COSC!) 


FIND: 
COST (M) 


PRINT: 
u(L),H(1) ,x(Z) ,cOosT(1) 


Z \ 
STOP 


Figure II-2: (concluded) 
6 


XV = 





if Lr "ya i hE = 


on 


a) 


START 


WV 


ep 
© 


A(1,J)=0.0 


Kip 


A(1,J)= ACigJ +B C1, K)*C(K,J) 


EXIT 


Figure LI-3: Flow Chart For Subroutine PROD(A,B,C,L,M,N) 


17 





START 


T+] 





Plead). = BiG) rel, J) 


Nigel 


Boer 


Figure II-4: Flow Chart For Subroutine SUM(A,B,C,N) 


18 





START 
WW 
Porm = A(1,J) 


< 2 
he < 
EXIT 


Figure II-5: Flow Chart For Subroutine TRANSQ (A, B,N) 





START 





EXIT 


Figure II-6: Flow Chart For Subroutine TRANCOL(A,B,N) 


START 





Figure II-7: Flow Chart For Subroutine Money (MON,E,Q,N) 


20 








CC = (QtP)* A 


EE = A!x*cc 
A=I 

BB = BB/EE 

BB = A —- BB 


» = BB*6 


EXIT 


Figure II-8: Flow Chart For Subroutine PPSI(B,C,D,E,F,N) 


Dal 








EXIT 


Figure 
II-10: 
: Flow C 
hart For Subrouti 
ine FF(A 
(5, CURE 
3 
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CHAPTER III 


PROGRAM OPCON2 


Program OPCON2 was written as the final phase for a versatile simu- 
lator to determine a constant gain matrix for optimization of a control 


system according to the following cost function: 


S(N)=minimum 2 Ge" (k) Qe(k)+ru? (k-1)) (45) 


Again u(k) is a scalar control and the Q matrix is determined in 
the same manner as outlined in the previous chapter. The variable, r, 
is a weighting constant, (Lagrange multiplier) determined by user needs 
and acts as a quadratic integral constraint on the control effort; such 
as a limit on energy necessary to provide the desired control. By let- 
ting r equal zero, we have the cost function of OPCON1 and identical 
results are obtained from the two programs. With the use of OPCON1, no 
restraints are put on the control required by the system which may re- 
sult in the necessity of an unlimited supply of fuel to obtain the de- 
sired optimal results. 
Again let us consider the system: 
xX = Fx + Du (1") 
whose discrete solution is: 
x(k+1)= ox(k)+Au(k) (4') 
where u(k) and A are defined in the same manner as in the previous chapter. 


By starting with the last stage first, equation 45 becomes: 


J (N)=minimum \x (N)Qx (N)+ru? (N-1)| +J (N-1) (46) 
or by substitution of equation 4': 


J (N) =minimum | (gx (N-1) +u(N-1))- Q (Gxe (NL) hu (ND 420? (N-1) +J (N-1) 
(47) 


203 





We now minimize J(N) with respect to u(N-1) noting that J(N-1) is not a 
function of u(N-1). 


oy JON 


u(NeT) = 0 = AlQox(N-1)+47Qdu(N-1)+ru(N-1) (48) 
On, 
Aloo 
u(N-1)= - = _ (49) 
ATQA+ x(N-1) 


As in Chapter II, the denominator of equation 49 will always be a scalar 
number; therefore, no matrix inversion is required. 

By proceeding back one more stage, the control policy u(N-2) may be 
optimized. Equation 45 becomes: 
J (8) =minimum fx (W) Qe () ta" (NL) Qa (8-1) teu? (HD tru? av-2) |40%-2) (50) 


The following symbology will be used for clarity in this minimization: 


A & 9x(N-2)+Au(N-2) (51) 
and, 
at 
eee (52 
A QAtr 


By direct substitution using 4', 51, and 52, equation 50 becomes: 


J(N)=minimum| ($A+ABA)™Q(A+ABA)+AQatr (BA) (BA)+ru2(N-2)}+5(N-2) (53) 
By rearranging terms, 
j (¥)=minimun]aTqa+ ( $+AB)A)*Q((o+AB)A) +x (BA) “+ru2 (N-2)} +J (N-2) (54) 
We now let, 
a, = B= - zat (55) 
ATQ a+r 
and 
v= o+da; (56) 
Equation 54 now becomes: 
FQN) =minimunl ATQA+ATY, QU, Ate (ajA)*4ru”(N-2)} +9 (N-2) (57) 


24 





We now minimize J(N) with respect to u(N-2), 


0 J(N 
an =0= ATP ataTy 1Qv, Ate a, Aa, Atru(N-2) (58) 


Solving for u(N-2) by substitution of 51 into equation 58, 


Jk 
u(N-2)= - PATQHAT YT QYy+rAT a, ay} ¢ 


x (N-2) (59) 
alk — 
{AT QA+AT yTQU, dtr a,4j A+r] 
We now let, 
ile iE 
P,=¥1QWitQ trajay (60) 
Equation 59 becomes: 
Dee ae 
Oe = x (N-2) (61) 
APP, Atr 
As we did in the previous iteration, we let, 
alt 
al.j AP)? 
2 re (62) 
Alp Atr 
= 
and, 
T 
bo= tha, (63) 


By proceeding back one more stage, a recursive relationship may be defined. 


We shall define the following quantity for brevity in this development: 


G & ox (N-3)+Au (N-3) (64) 
The cost function, 45, may be written as, 
JN) minimum 3 (8) QCM) Aa (WA) GCL) Hae (HH 2) ECE 2) 
tru? (N-1) tru? (N-2)+ru2(0%3)] HI CNS) (65) 


By substitution of 43 55, 56, °62, Gaemana os, equation 65 becomes: 


Zo 





J (N)=minimum \ (¥2¥16)TQ(¥201G) +r (¥241 G6) 1 (H24;7¢) 
T T 
+(¥oG) G(poG)tr (ay Ore G) 
+oT9Gtru? (n-3)} 45 (N-3) (66) 
Minimizing the cost over N_ stages with respect to u(N-3), 


9 SO) ee ra T 
u(N-3) = 9 = 44,4, Qh, ¥,Gt+4 ¥,QH,6+4 Qc 


Lr Ti li a 
ae - +rA -} = 
rA ye ae r TS ru (N-3) (67) 


Substituting 64 into equation 67 and solving for u(N-3), we find the 


optimum control policy: 


u(N-3) = 


{ aTyZyTou,y +oTyZQu +iTqeraT ula, aly trata,’ } ¢ 


eu 0 +ATUTOUM REO EC Eee. aTan) 2c(Ne3) (68) 
{a Do ee 5 rt 
Combining terms, 
DD iT T Tee 
u(N-3)= - {AT Wo (VIQH]+0+raj al) Vote QtrA ara, | p 
x(N-3) (69) 


0 T eel 
| oT v5 (WE Qu: +G+ra1a]) by dth QAatrA araodtrt 
From equation 60, equation 69 becomes: 


_ _ § Alwi pe Wipe one ees | 
u(n-3)= - {4 ¥2Pi¥ot Gerd aa, }¢ x (N-3) (70) 


Te T 1 Je 
{ A'voP] v2AtA'Qdtr A aya, br} 


We now let, 


_ 2 iT 
Po=WoP botQtra,a, (71) 
and equation 70 becomes: 
al , 
u(N-3) = - SEF 3Gr3) (72) 
eee 


Let, 


26 





We al 
oo eit? 
Alp A+r 
a 
and, 


ik 
= gtda 


v 3 


3 


(73) 


(74) 


As in Chapter II, a recursive relationship may now be shown without 


proceeding back another stage and may be shown to be true by an induc- 


tive proof. Let r 
ae = = A Pu-1? ) 
ATP, _, dtr a 


and, 
» = $a » Y=0 
K alas O 
where, 
PS Teeter ep) = 
ko He ee ate eee FO 
and, 
ela Sue Gne) 
i 
from, 


x(ktl)= 9x (k)tAu Ck) 


By letting r equal zero and solving the same illustrative 


2) 


(76) 


(77) 


(78) 


ie) 


example as 


in the previous chapter, the same results are obtained as illustrated 


below. 
i 
T. =—A O78 
a, | L2.0 2.0] 
A QAtr 
and, 
0 0 
W = p+da, = 

l 2) =e) 

and, 


Za 


(80) 


(81) 


(82) 





and therefore, 


u(k)=a, x(k)= - 2x, (k)=2x, (k) (83) 


Equation 83 indicates the same results as in Chapter II and perscribes 
the same phase plane (Figure II-1). 

Through the use of the recursion formulas 75, 76, 77, 78, and 79, 
PROGRAM OPCON2 was written. Figures III-1 through III-5 and Figures II-3 
through II-6 represent abbreviated flow charts of the program and the sub=- 


routines used in the program. 
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START 






READ AND PRINT 
Pile. A, 
$,Q,and X © 


KN= 0 
| TCOST= 0.0 


* » 


(I,J) 


a 


al 


< > 


PIL, De. 0 
Sia 
> < 


o 


al 


Figure III-1: Flow Chart for PROGRAM OPCON2. 
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FIND: 
y and P 





AT,w, and P 





OZ 


Figure III-1l: (continued) 
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P1(J,K)=P(J,K) 


AT(1,1)= ALFAT(1,1) 
? r4 


a3 


Figure III-1: (continued) 


ot 


Bl 








FIND: 
DOL 
TCOST = TCOST+DOL 


a4 


Figure III-l: (continued) 
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a4 


PRINT: 
L,Z,X(1), and TCOST 







YQ, 1) =e (J, 1) 


DELTA (J,1)=Z*DEL(J, 1) 
K(J,1)= X(J,1)+DELTA (J, 1) 
AT (1s) =AunA Ee Gid) 


a5 BZ 


Figure III-1: (continued) 


3 3 








AT(J,K)= 


Figure Iil-1: 
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0.0 


(continued) 


B2 


B2 





a6 R2 








PRINT: 
I,Z,X, and TCOST 


Figure III-l: (continued) 


ao 





START 


FIND: 7 


| 


AB= A°*P 


AC= AB*A 
AD= AB*¢ 






AT(1,1)= =ADG@iye), AG (1,1)+R) 


a 


EXIT 


Figure III-2: Flow Chart for SUBROUTINE ATRAN (AT, P,PHIL,DEL,R,N) 


a6 


tt 





START 





EXIT 


Figure III~3: Flow Chart for SUBROUTINE PPSI(PSI,PHI,DEL,AT,N) 


START 


=e 
= A*Y 
AA(1,1)+R*Z4Z | 





EXIT 


Figure III-4: Flow Chart for SUBROUTINE COST (DOL,Y,0,R,Z,N) 
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START 








J 71S< 


AD(1L,J)= R*AD(I,J) 


2 
o/ 63 


Figure [II-5: Flow Chart for SUBROUTINE PP(P, PSI, Pil GyeAi Reap 
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a7 B3 






AE= AC+Q 
P= AE+AD 


Figure III-5: (concluded) 
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Figure III-l shows that the inputs to the program must be read from 
data cards. Table III-1 shows the order in which these cards are read, 


the format required on each card, and the definitions of the variables 


concerned. 


Variable 


Defini eron 


Number of stages 
Plus one 


Order of system 


Frequency of the 
desired printing 
of output 


Weighting constant 
of cost function 
Column vector of 
system impulse 
responses due to 
control inputs 


State transition 
matrix 


Weighting matrix of 
the cost function 


Initial conditions 
on the state vector 


Format of 
Data Card 


(read from 
bottom of column) 


nF10.6 
(read in a 
a time) 


nF10.6 
(read in a 
a time) 


8F10.6 
(read from top to 
bottom of column) 





Table III-l1: Definitions and Format of Data Cards for 
PROGRAM OPCON2. , 
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The output of PROGRAM OPCON2 is the value of each state variable, 
the control signal, and the total cost of all previous stages includ- 
ing the present stage. The amount of printed output may be varied by 
the use of the IT variable's input data card. If all output is to be 
printed, then IT must be set to one by the data card. If the user 
wishes to have 1,000 stages and has a requirement of print out of 
only a sampling of these stages, IT should be set accordingly. By let- 
ting IT equal ten, only every tenth stage will be printed out. Another 
valuable output is the value of a which approaches the constant gain 
matrix necessary for optimal control of a continuous system. This is 
achieved by using a small sample period and many iterations thereby 
approaching the simulation of a continuous system. OPCON2 is limited 
for this use because the computer memory size limits the program to ap- 
proximately 2,000 stages. If the user has no need for the outputs of 
the values of the control signals, state variables and total costs, 
then the program is unlimited in its number of stages because the value 
of a may be printed out as often as desired without storing it in mem- 
ory. This alteration has been made in PROGRAM OPCON3, and a listing 
of this program may be found in the appendix. 

The versatility of this program may be further shown in using it 


e 


to simulate an optimum control in accordance with the cost function, 
J (N)= minimum] X7 (N)X (8) (84) 


which we shall call the final value criterion. Using the same method 


of derivation as before, we find the following recursion formulas: 
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and, 


where, 


and, 


=o+A : = 0 
Ve d day ue 
a 
T . _ 3a > jae 
ae - K-1 a =0 
tigers 


T 


a P P21 
Kel Yel K-2"K-1 O 


a = AE — 
u(N-K) a ZN K) 


(85) 


(86) 


(87) 


(88) 


To use OPCON2 for this cost function, we need only to set P equal 


to the identity matrix, Q to the null matrix, and r equal to zero. 


As an example of this cost function, we shall again use the same 


illustrative system. We find that: 


and, 


and, 


and, 


therefore, 


2 | 
a) See 4 -1.2] 
1 T 





AT 
viata = 8 4 
-.4 -.2 
7. 
a 4) aT 8 4 
me Z 
- 
= = =-1 = [1 0 a3 
A PUA 
u(0) = - X, (0) = 1.5X, (0) 
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(89) 


(90) 


(7) 


(92) 


(93) 





Given initial conditions of X, (0) aol O earl X. (0) = 0.0, equation 
o3uresults Ain, 
noe — 1.0 (94) 


therefore, 


X,(1) = X,(0)+0.5u(0) = 0.5 (95) 

and, 
X,(1) = X,(0)+u(0) = - 1.0 (96) 

From equation 89, 

u(1) = -.4X,(1) - 1.2X,(1) = 1.0 (97) 

and, 
K, (2) = x, (1)4K, (1)+.5u(1) = 0.0 (98) 

and, 
X, (2) = X,(1)+u(1) = 0.0 (99) 


Figure III-6 is the phase plane resulting from the calculated 
controls and shows that the final states at N equal two are definitely 


at a minimum and the cost function equals zero. 


X 





Figure III-6: Phase Plane for Example Problem. 


43 





With the use of OPCON2 and OPCON3, many control systems may be 
simulated. Using these two programs, a system may be designed for a 
number of optimization criteria while greatly reducing design time. 
In the next chapter, a typical system will be considered assuming 


an optimization criterion and the resulting design will be obtained. 
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CHAPTER IV 


EXAMPLE PROBLEM 


To show the use of the programs that were written and developed 
in the preceding chapters, the following example problem was chosen. 
The transfer function is typical of a servo motor, the roll attitude 
control of a missile, and many others. Figure IV-l1 shows the open 


loop plant to be controlled. 


gels i fel 
S+1 S 


Figure IV-1l: Block Diagram of the Open Loop Control System. 


The following state equation defines the system: 


x= x + | lu (100) 


With the use of PROGRAM PHIDEL and a sample period of one second, $4 


P00 652 . 368 
x(kt1l) = auGks) ct: u(k) (101) 
UeU 368 OZ 


To show the flexibility of PROGRAM OPCON2, four different cost 


aindeo are: 


functions will be considered and compared. The first cost function 


will be the terminal value function for time optimal control. (Case I) 
J(N) = minimum x! (N)x(N) (84') 


This will then be compared to the terminal value control with 


an energy control shown by equation (102). (Case II) 
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k=N 
HCN) Sonica US (KT) (102) 
k=1 


The next case of interest minimizes the sum of the squares of the 


states with an energy limitation. This is represented by equation (45'). 


(Case IIT) 
k=N r 
J(N) = minimum > x (K)Qx(K)+ru2(K-1) Gey) 
k=1 
where, 


QO = _ieand: r 


iO (103) 


The final cost function will also be that of minimizing the states 
with an energy limitation; however, by use of PROGRAM OPCON3, a steady 
state feedback matrix was found. These values were then used in 
PROGRAM OPCON2 to obtain a transient solution. (Case IV) The elements 


Gempnis Control. are: 


a (1,1) = -0.627797 (104) 
elacl. 
ab (1,2) = -0.615037 (105) 
With the use of PROGRAM OPCON2, a transient solution was obtained 
in each of the four cases. Case II was computed for a twenty second 
trajectory in the phase plane, Figure IV-3. Cases III and IV were each 
run for ten second trajectories while Case I was optimized in two samples. 
Figure IV-2 is a plot of control versus time for each of the cases. 
It is of interest to note that both cases III and IV utilize the same 
controls during the first seventy percent of their trajectories; only 
when the control is very small in the last couple of sampling points 
does the control required differ. This thereby leads to identical trajec- 


tories in the phase plane indicating that constant gain amplifiers in the 
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feedback loop are just as effective as variable gain amplifiers for com- 
pensation of this plant. Case II requires nearly a constant control ef- 
fort after the initial control jump has been made. The required control 
for the twenty second trajector is approximately equal to one half of 
the necessary control for a ten second trajectory. 

Figure IV-3 is a phase plane plot of the four cases. One thing to 
note here is the trajectory of case II. As the total number of sample 
periods is increased, the necessary control is decreased as indicated 
above. This results in the trajectory approaching the xy axis as the 
number of sample points increases indefinitely. This means that the 
cost function used is optimized when the final value of the states is 
zero and the control is nearly zero. The point in the phase plane would 
drift in requiring only a very slight amount of control to stop it at its 


steady state value. 
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am 
<2 
a 
ep a 
> = i] 
a, 
=> 
ng ~ 
' 
a 


=> 

ade 7 

ae 

- 
‘ = 

. 

6 = = 

' 

ed 





DO 





2.0 YO S.0 LAO HLLO 


Figure IV-2: Plot of Control Versus Time for the Four Cases. 








= Can 


Figure IV-3: Phase Plane Plot of the Four Cases. 
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